In a typical multi-media computer system, a conventional personal computer is augmented with a full-motion video card to display full-motion video images in a window on its display screen. The video information may be generated by any source. Still images, such as text, may be concurrently displayed outside the video window. Such a system offers enormous potential for educational applications and other interactive applications.
The full-motion video data may be recorded on a hard disk or other storage medium used by the computer for later playback and display. When attempting to store full-motion video data on a hard disk, a problem exists in that the bandwidth of the original full-motion video data is much greater than the bandwidth of the conventional hard drives used in personal computers. Further, the unpredictable track access times for recording on a hard disk cause the short-term bandwidth of the hard drive to be unpredictable. Accordingly, the complete original video data cannot typically be saved on a hard drive for later playback.
The system of FIG. 1 will be used to illustrate this problem and other problems with the existing art when attempting to store full-motion video data on a hard disk using a personal computer. The original full-motion video data may be in the form of a standard analog video signal such as in a National Television System Committee (NTSC) format or other format. Assuming the original video data is analog, this analog signal is then converted to a digital signal using an A/D converter 10. This digitized signal is then applied to an input of a full-motion video controller card 12 (which may or may not include the A/D converter 10). Such a video controller card 12 includes a full-motion video memory 14. One type of video controller card and video memory is described in U.S. application Ser. No. 08/136,621, previously mentioned.
The video controller card 12 is connected to a system bus 16. Also connected to bus 16 and to an output of video controller card 12 is a conventional VGA controller and frame buffer 20. The full-motion video data contained in the video memory 14 is multiplexed with the data stored in the VGA frame buffer 20, and this multiplexed data is applied to a monitor. The monitor then simultaneously displays a full-motion video window along with other data outside the window.
The personal computer's CPU 22 is connected to the system RAM 24, and the CPU 22 is connected to bus 16 for controlling, among other things, transfers between the system RAM 24 and the hard drive 26.
The conventional hard drive 26 is connected to bus 16 via a conventional buffer 28. A data transfer from RAM 24 to hard drive 26 is asynchronous with the operation of the video controller card 12. Data is transferred to hard drive 26 by transmitting bursts of data over bus 16, and buffer 28 is used to temporarily store this burst of data while the hard drive 26 records the data on a rotating disk, consuming an unpredictable amount of time. Buffer 28 may have a capacity on the order of 64 Kbytes.
FIG. 2 illustrates a conventional operation of the system of FIG. 1 when storing full-motion video data on the hard drive 26. The full-motion video data is conventionally transmitted as frames of data, where each frame stores all the pixel information needed to display a single image on a monitor having, for example, 640.times.480 pixels. A typical frame rate is 30 frames per second. When this pixel data is digitized, the rate of pixel data may be on the order of 20-30 Mbytes per second, depending upon the digitizer used.
In a conventional NTSC format, each frame is composed of two fields of interlaced lines of pixels, so that each field has a period of approximately 1/60th of a second. The fields may be referred to as even fields and odd fields since one field energizes even lines of pixels on the monitor and the other field energizes odd lines of pixels on the monitor. This assumes a raster-scan type monitor is used.
Due to the extremely large bandwidth of the incoming digital video data, the video data is selectively scaled down by circuitry on the video controller card 12 to reduce the bandwidth of the data to make this bandwidth compatible with the bandwidth of the video memory 14, the bus 16, and the hard drive 26. The bandwidth of the hard drive 26 may be on the order of 400-600 Kbytes per second, which is similar to the bandwidth of bus 16. Thus, the original video data must be scaled down considerably, both horizontally and vertically, to reduce its average bandwidth to 600 Kbytes per second or less.
FIG. 2 illustrates five time periods T1-T5 during which time full-motion video is being stored in the video memory 14 and subsequently transferred to the hard drive 26. A single port video memory 14 is presumed, although the problem being addressed would also arise if a dual-port VRAM were used as the video memory 14. Each time period T1-T5 coincides with the transmission of an even or odd field of video data.
During time T1, an even field is transmitted, scaled down, and stored in the full-motion video memory 14 in FIG. 1.
At time T2, the even field video data already stored in the video memory 14 is transferred via bus 16 to the system RAM 24. This step is necessary since all data to be transferred to the hard drive 26 must be first stored in the system RAM 24 and processed by the CPU 22. During this time T2, the odd field of video data being transmitted by the video source is being dropped to free up the video memory 14 for the data transfer to RAM 24.
At time T3, a next even field of video data is stored in the video memory 14. During this time, the bus 16 is free, and the video data previously transferred to the system RAM 24 is now asynchronously transferred via bus 16 to the hard drive 26 for storing the even field received by the video memory 14 at time T1.
This process of storing and transferring is then repeated in an attempt to capture one field per frame of full-motion video data transmitted. During this time, the data stored in the video memory 14 may be displayed on a monitor without requiring the use of the bus 16.
Due to the inherent recording time unpredictability of hard drive 26, the required time to fully transfer a field of video data between the system RAM 24 and the hard drive 26 may exceed the allocated period T3. Such unpredictability stems from the varying track/sector access times for the hard drive 26 and the mechanical timing variances inherent in the hard drive 26.
Since the bus 16 is used to transfer data from the system RAM 24 to the hard drive 26 and to transfer data from the video memory 14 to the system RAM 24, the consecutive transfer steps shown at times T2 and T3 cannot overlap since they both require the use of bus 16. Thus, if the time for transfer of the data from the system RAM 24 to the hard drive 26 during period T3 is greater than 1/60th of a second or otherwise overlaps the time period T2 or T4, a complete frame of video data will be lost. Due to the unpredictability of the timing and bandwidth of the hard drive 26, unless large margins are provided between the memory transfer steps in time periods T2 and T3 (and subsequent periods) to account for worst case access times by the hard drive 26, frames of video data will be lost at random. Due to this random loss of frames, when this video data is later played back after capture on the hard drive 26, the displayed moving video image will be erratic and of limited value.
The prior art has attempted to avoid this random loss of frames by either greatly reducing the bandwidth of the video data (i.e., dropping large numbers of pixel bits) to reduce the time it takes to transfer one field of data over bus 16, or periodically deleting frames (e.g., delete every third frame) to reduce the frame rate and, hence, increase the time period allocated to transfer a field or frame of video data to the hard drive 26. Thus, the prior art systems are designed around the worst case bandwidth (i.e., minimum peak bandwidth) of the hard drive 26. Such prior art attempts to create reliable and predictable video storage and reproduction are at a great cost to the resolution and/or fluidity of the subsequently displayed video image.
What is needed is an improved video processing technique which does not require as great a reduction in bandwidth of the video signal or frame rate reduction as prior art techniques and which may be implemented with a minimum of additional hardware over that described with respect to FIG. 1.